Field
The disclosed embodiments relate to techniques for processing network data. More specifically, the disclosed embodiments relate to techniques for extracting custom content from network packets.
Related Art
Over the past decade, virtualization has triggered a sea change in the field of network data capture. Almost every network capture product available today is a physical hardware appliance that customers have to purchase and configure. In addition, most network data capture technologies are built from scratch to serve a specific purpose and address the needs of a particular market. For example, network capture systems may be customized to extract data for security and intrusion-detection purposes, collect network performance data, facilitate Quality of Service (QoS) policies, redirect data, block network traffic or perform other analysis or management of network traffic. However, such targeted and fixed implementations of network capture technologies are not designed to be easily modifiable to address different business needs.
One challenge in generalizing a network capture system is to handle large volumes of network data. A network capture system typically monitors a large number packet streams at various locations in a network. Moreover, each packet stream can potentially run at gigabit clock rates, and as a consequence can generate tremendous volumes of data. It is theoretically possible to store all of this network data for subsequent analysis. However, as a practical matter it is infeasible and prohibitively expensive to store such large volumes of network data. Also, much of the data contained in packets is uninteresting; only specific packet fields contain interesting data for a particular purpose.
Hence, what is needed is a system for capturing network data that facilitates selectively retrieving specific data from packets.